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Doscriptio nT ITLE OF THE INVENTION 
Virtual assistant, which outputs audible information to a us e r of a data terminal b y 
m e ans of at l e ast two e l e ctroacoustic conv e rtors, and m e thod for pres e nting audibl e 
5 information of a virtual assistant 

VIRTUAL ASSISTANT AND METHOD FOR PROVIDING AUDIBLE 
INFORMATION TO A USER 
BACKGROUND OF THE INVENTION 
The invention relates to a virtual assistant, which outputs audible 
10 information to a user of a data terminal by means of at least two electroacoustic 
converters, and a method for presenting audible information of a virtual assistant 
for a user of a data terminal. 

When using PC application programs, it is generally known that the user 
can make use of a virtual assistantT4ba t. A virtual assistant i s to say a computer- 
15 based help (program) program that supports the user when carrying out the steps 
necessary to perform a taskr-e f on the computer. The virtual assistant may also be 
invoked when the user wishesneeds further explanations about the capabilities of 
the PC application program. S e condlv, T he virtual assistant may also direct the 
user's attention is drawn to any incorr e ct inputs and the virtual assistant malc e s 
20 inpu tto any input mistakes the user makes and may make suggestions to the user. 
The information provided by the virtual assistant is presented to the user 
opticall w isuallv. that is to say by means of a display unit. 

In principle, this functio nt he functions of a virtual assistant which isare 
helpful to tfiea user can also be applied to mobile data terminals such as mobile 
25 phones or handheld t erminals that are known as Personal Digital Assistants 
(PDAs). In this case, however, it is disadvantag e ous for th e user that t he extensive 
information pr e s e nt e d by the user of visual data by a traditional virtual assistant 
must b e displayed on a is a disadvantage due to the small display unit of the mobile 
data terminal. 

30 Moreover, the e xtensive amount of information e fpresented visually by a 

virtual assistant that is presented optically is difficult for the user of a handheld data 
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terminal to process when e v e r th e us e r n ee ds to in situations where the user must 
concentrate on other opticall\ ^ isuallv presented information presented in the same 
vicinity or on acoustic information ef- asimultaneouslv presented such as an 
ongoing conversation partner at the sam e tim e w ith an associate . In this case it is 
5 expedient to provide the information presented by the virtual assistant of a data 
terminal fe fto the data terminal user by means of an acoustic presentation. In this 
way, the data terminal user can bette miore easily process the acoustically presented 
information and — additional along with the other information optically 
pres e nt e db eing simultaneousl v presented either visually or accousticallv . 

10 On th ein other haa dapplications . data terminals or methods are Icnown in 

whieh emploved where additional information is acoustically presented to the user 
of the data terminal or of th e method accoustically . For instance, an audio assistant 
in a ticket machine guid e s th e may be used to guide a user of the ticket machine 
through the respective operating programs of the ticket machine by m e ans of 

15 acoustic information . Sinc e thes e However, such ticket machines and like devices 
are often sited in a loud environment, it is noisy environments. It is often difficult 
for th e us e r users of the ticket machine to feHewhear the acoustic information 
output by the audio assistant of the ticket machiner- ft and follow the instructions 
being presented. 

20 An additional complicating factor in presenting acoustic information is that 

it is even more difficult to follow acoustic information that is simultaneously acting 
on a user fi"om two different signal sources. So-called binaural technology has been 
the subject of research for some time now. For example, an introduction to binaural 
technology is described under the title: "An introduction to binaural technology" by 

25 J. Blauert (1996) in Binaural and Spatial Hearing in Real and Virtual 
Environments, edited by R. Gilkey & T. Anderson, pages 593-609, Lawrence 
Erlbaum, USA-Hilldale NJ. 

With the aid of binaural technology, using signal processing of the sound 
informatio n can be employed to give , the listener can assig nt he sense that the 

30 sound-generating sourc e is assigned to any positions o fp osition within the 
surrounding space. Th e positio n Though the relative positions of the listener-iiCTej 

13 



and of the electroacoustic converters outputting the acoustic information 
rospoctiv e ly, remains remain spatially fixed . By m e ans of suitabl e s ignal proc e ssing 
of the sound information, it is th e n possible for e xampl e , it is possible to awaken in 
the listener the subjective impression that the sound-generating source is tuming 
5 around him, or is oomin gm oving toward him, w-4s-moving away firom him r, or 
changing in some other way. By signal processing of the sound information, the 
sound-generating source can th e r e for e b e spatially positioned anywhere. It is 
ther e for e the object of th e pr e sent invention to develop a tochnieal solution for the 
us e r of a data t e rminal, in which th e acoustic information output by the virtual 
10 assistant of th e data terminal can b e bett e r separated, in terms of the user's 
perception, from other sound sourc e s that ar e lik e wis e acting on th e data t e rminal 
ttse fbe positioned anywhere in space, vet give the impression to the user that it is 
located elsewhere . 

15 The object is achieved on th e basis of th e virtual assistant defin e d in th e preambl e 
of claim 1 by th e f e atur e s set out in th e characterizing part of claim 1, and on th e 
basis of the method d e fin e d in th e pr e amble of claim 9 by the features sot out in th e 
charaoterizing part of claim 9. Advantageous r e finem e nts of th e inv e ntion ar e s e t 
out in th e subclaims. 

20 SUMMARY OF THE INVENTION 

According to the invention, a virtual assistant which outputs audible 
information to a data terminal user by means of at least two electroacoustic 
converters can be spatially positioned by the user in order to achieve a better 
spatially acoustic separation between the information output by means of the 

25 electroacoustic converters and additional information output by at least one 
fiirth e r other sound source. 

OneAn advantage of the invention is the utilization of th e spatial positioning 
of sound sourc e s bv means o fi s that signal processing of the sound information of 
the virtual assistant e finay utilize the spatial positioning of the sound sources of 

30 relative to the data termina l, or its locating by the data terminal us e r r e sp e ctiv e ly. 
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For th e data t e rminal us e n said sound information e f user so that the virtual 
assistant can be better perceived separately from ambient neise snoise . 

Furthermore, the sound information of the virtual assistant can be supplied 
to the data terminal user in a targeted manner from eft ea specific direction, while 
5 the user is simultaneously holding a conversation with another conv e rsation 
portne r someone else in the room. Here, too, it is possible to achieve a 
^gee dsatisfactorv spatially acoustic separation between the sound information acting 
on the user from the virtual assistant and from the conv e rsation partn e r person 
conversing with the user . This enables the user to receive and process both the 

10 information coming from the virtual assistant and t he information coming from 
feehis conversation partner. The simultaneous r e c e ivin gr eception and processing of 
both fe esets of information coming from th e virtual assistant and th e information 
coming from the conversation partner is how e ver ig at least facilitated for the user. 
A further advantage emerges when, in addition to the soxmd information 

15 coming from the virtual assistant and the ambient noises originating from 

feffee fother sound sources present in the vicinity of the user, visual information is 
also information that is additionally optically p resented simultan e ously acts ont o 
the data terminal use r at the same time . In this case, too, the data terminal user can 
better receive and process the information coming from the various sources. Furthor 

20 Additional features and advantages of the present invention emerge from 

the description below, in which the inv e ntion is explain e d with r e f e renc e to two 
ex e mplary e mbodim e nts, a re described in. and will be apparent from, the following 
Detailed Description of the Invention and the figures. 

25 DETAILED DESCRIPTION OF THE INVENTION 

In thea firs t exemplary embodiment, a pedestrian is situated in road traffic. 
The pedestrian is laden with heavy shopping bags. The pedestrian would like to 
conduct a phone call using his data terminal in the form of a mobile phone. The 
mobile phone is switched on, but is stowed away in one of his shopping bags and 

30 therefore cannot be readily located. The pedestrian is wearing a light headphones 
and microphone set however. Integrated in the headphones and microphone set are 
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two electroacoustic converters for outputting sound information. Like the mobile 
phone, the headphones and microphone set is connected to a radio module, for 
example to a Bluetooth radio module, for short-range data exchange between the 
headphones and microphone set and the mobile phone. 
5 The pedestrian, user of the headphones and microphone set and of the 

mobile phone respectively, activates the headphones and microphone set and thus 
enables data exchange between the headphones and microphone set and the mobile 
phone. The user speaks the word "DIAL" into the headphones and microphone set, 
whereupon the virtual assistant of the mobile phone responds with "PLEASE SAY 

1 0 THE NAME". The user says the name of the person he wishes to call. Since the 
user is moving in an environment with a high noise level, the mobile phone does 
not recognize the name of the person to be called with sufficient accuracy. The 
mobile phone processes the name entered by the user and compares it with names 
stored in the intemal phone directory of the mobile phone. The mobile phone 

15 recognizes the name spoken as "SCHMITZER" or "SCHNITZLER". Output of the 
two names to the display unit of the mobile phone and the subsequent request to the 
user to select one of these names is of no use to the use r. This is because, as already 
mentioned, the user's mobile phone is hidden in one of the pedestrian's shopping 
bags in a place that is difficult to access. On th e other han d However, the mobile 

20 phone has recognized the op e ration of the mobile phon e r equest by the user via the 
headphones and microphone set, so the mobile phone instracts the virtual assistant 
of th e mobil e phon e t o output all similarly sounding names to the user by means of 
th e virtual assistant using the headphones and microphone set. For example, the 
user hears the following words of his virtual assistant via the headphones and 

25 microphone set: "THE NAME WAS NOT CLEARLY RECOGNIZED". "PLEASE 
SELECT ONE OF THE FOLLOWING OPTIONS". "SCHMITZER" or after a 
brief pause "SCHNITZLER". 

Despite the loud ambient noises, the user recognizes both the options 
offered by the virtual assistant because binaural technology is used during the 

30 output of the soimd information of the virtual assistant of the mobile phone by 
means of the electroacoustic converters. The binaural technology enables the 
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targeted signal processing of the sound information i aoutput by the mobile phone. 
When the sound information is played back by the virtual assistant using the 
headphones and microphone set, the mobile phone user can perceive a clear local 
attribution of the sound information output by the virtual assistant. In accordance 
5 with a user preset^ in the mobile phone^ the soimd information is processed using 
signal technology in such a way that the mobile phone user locates the sound 
information presented by the virtual assistant as if it were coming from the vicinity 
of the head. The sound information is "whispered" into the user's ear over his 
shoulder from behind. 

10 The position of the virtual assistant, or the position from which the soimd 

information output by the virtual assistant is perceived respectively, can be changed 
as desired by the mobile phone user, for example by means of an electromechanical 
input device as is well k nown pef ^in the art . 

The electromechanical input device i smav be for example a ball zin-a- 

1 5 socketT-Th e input device, where the rotations of the ball produced by the user are 
detected by sensors. Altematively, the positioning of the virtual assistant is mav be 
performe d in a manner Icnown per s e by means of voice commands or by means of 
inputs on a touch-sensitive display imit of the mobile phone. 
If the mobile phone has a head position sensor which detects the head movements 

20 of the mobile phone user, for example using a rotational rate sensor or a magnetic 
field sensor, it is furthermore possible for the selected position of the virtual 
assistant to be retained even if the head movements are taken into account during 
the signal processing of the sound information. 

By means of the preset positioning of the virtual assistant, or resp e ctiv e ly 

25 the ability of the user to change its position as desired, the user can both operate the 
mobile phone in a simple manner using voice commands to establish an outgoing 
connection as well as attentively perceive ambient noises, such as loud calls o r the 
sovmding of horns etc. 

To finish the selection of the names "SCHMITZER" or "SCHNITZLER" 

30 presented by the virtual assistant in order to establish an outgoing connection, the 
user responds to the name "SCHMITZER" by speaking a "NO" into the headphones 

17 



and microphone set and by responding "YES" for the name "SCHNITZLER". The 
mobile phone recognizes the name "SCHNITZLER" and estabhshes an outgoing 
call. 

In thea second e x e mplary e mbodiment, a t e l e conf e r e ncing situation io 
5 describ e d. Talcing port in the t eleconference ar eis established among a plurality of 
peopl e who for the most por t , many of whom speak and understand different 
languages. The pee ^participants in the teleconference are situated in e ach cas eat 
individual tables spread throughout thea teleconferencing roo m, with e ac h . Each 
person havm ^as their own display. If one participant starts to speak, th e n th ea data 

10 terminal in the form of a teleconferencing system displays feisthe participant on a 
large screen on a side wall of the teleconferencing room, so that the other 
participants can alse-observe the facial expressions and gestures of tiasthe 
participant who is speaking . 

Secondly, hi sthe speaker's speech is output via electroacoustic converters in 

15 the form of loudspeakers which are connected to the teleconferencing system. 

At the same time, the contributions of the sp e alcing participant ar e speaker' 
speech is simultaneously interpreted into the languages of the other participants^ 
and th e translation is . The translations are made available to the participants in the 
form of sound information via a-headphones and microphone setsets in which two 

20 electroacoustic converters for outputting sound information are integrated. To offer 
the participants the option of attentively following the speech both in the language 
of the participant speaking and in the language of the simultaneous interpretation, 
the simultaneous interpretation is output by the teleconferencing system using a 
virtual assistant so that the other participants can hear it. The virtual assistant can be 

25 positioned anywhere in the room by each teleconference participant by entering the 
respective key combinations into the teleconferencing system. 

Here, too, the positioning of the virtual assistant, or the spatially acoustic 
perception of the sound information output by the virtual assistant by the individual 
participants respectively, is achieved by means of signal processing of the sound 

30 information in the teleconferencing system. The participants position the virtual 
assistant in such a way that the participants perceive the output of the sound 
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information by the virtual assistant as being transmitted over the shoulder from 
behind and coming from the vicinity of the head. By virtue of this positioning of 
the virtual assistant, a good spatially acoustic separation between the speech 
transmitted via loudspeakers and the simultaneous interpretation of the speech is 
5 achieved , so that th e . The participants can readily follow both the speech 
transmitted via loudspeakers and the simultaneous int e rpr e tation, and can 
stit ltranslation while attentively obs e rv e observing the facial expressions and 
gestures of the participant speaking. That is to say, the participants can4H^ily 
attentively follow a plurality of information streams at the same time. 
10 If one participant already knows what one of his own delegation is going to 

say, then said participant can have the teleconferencing system acoustically give 
him fiirther information via the virtual assistant, for example about the schedule for 
the day, backgroimd information about the other participants, or information about 
the participant's hotel. 

15 The above embodiments of the invention are merely examples givenand are 

not exhaustive. The concept of the— spatially acoustic separation and signal 
processing of sound information which is output to a data terminal user via a virtual 
assistant and additional simultaneously audible and/or visible information which is 
important to the user can be applied to fiirther examplesr4n:,^_In particula r, the 

20 present invention mav also be employed in cases where mobile communication 
terminals are employed by a user. Travel guides are cited here by way of example, 
wherein the travel guide explains certain exhibits of a museum to visitors in the 
local language of the countr vm usevun : the visitors are able to listen via th e ir UMTS 
mobile phon e t o a simultaneous int e rpr e tatio nt ranslation of the explanations of the 

25 travel guide wk hon their UMTS mobile phone having good spatially acoustic 
separation via a virtual assistant s and optionally . Optionally the user can attentively 
follow additional optical information relating to the exhibits on the display xmit of 
their UMTS mobile phone at the same time. 
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ABSTRACT OF THE DISCLOSURE 
A virtual assistant is provided which outputs audible information to a user 
of a data terminal A method is also provided for presenting audible information of 
a virtual assistant to the user of a data terminal. At least two electroacoustic 
5 converters are driven such that the virtual assistant can be spatiallv positioned by a 
data terminal use to achieve acoustic separation between the electroacoustic 
converters and an additional sound source. 
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